.

Chatting with our data

** Lets meet a very nice guy: dplyr **

• Maybe, the most of our time, will be spent on talking to our data

• One of the most important type of words on our languages are the verbs

• This make sense because they execute actions, and that is what we’ll want

1. We are going to need take some variable

2. and make a new one from it

3. Perhaps, there are going to be sometimes when we would like to know the possible cases that would come from the combination of two categorical variables and calculate the frequency of each scenario (distribution) and evenresume the information than’d come from it

4. Another times, we’d just need to order our dataset from some specific variable or variables in order to watch the biggest (or the smallest) ones at above (or at the bottom)

5. And finally what we’d want to tell our data is that we only want to see those cases (rows) than would fulfill some condition

So, translating …

Quoting from - dplyr’s github:

dplyr is a grammar of data manipulation, providing a consistent set of verbs that help you solve the most common data manipulation challenges:

1. select() picks variables based on their names.

2. mutate() adds new variables that are functions of existing variables.

3. summarise() reduces multiple values down to a single summary.

4. arrange() changes the ordering of the rows.

5. filter() picks cases based on their values.

# library("nycflights13")
# saveRDS(flights,"data/flights.rds")
# saveRDS(airlines,"data/airlines.rds")
library(tidyverse)

── Attaching packages ────────────────────────────────────────────────────── tidyverse 1.2.1 ──
✔ ggplot2 3.1.1       ✔ purrr   0.3.2  
✔ tibble  2.1.1       ✔ dplyr   0.8.0.1
✔ tidyr   0.8.3       ✔ stringr 1.4.0  
✔ readr   1.3.1       ✔ forcats 0.4.0  
── Conflicts ───────────────────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()

# library("nycflights13")
# saveRDS(flights,"data/flights.rds")
# saveRDS(airlines,"data/airlines.rds")
library(tidyverse)
mpg

select

# Drop unimportant variables so it's easier to understand the join results.
flights %>% select(year:day, hour, origin, dest, tailnum, carrier)

mutate

flights %>% summarise(I_tarde=if_else(arr_delay>0,1,0))

Error: Column `I_tarde` must be length 1 (a summary value), not 336776

arrange

filter

sum(is.na(flights2$I_tarde)); print(paste0("The 'NA's represent the: ", round(sum(is.na(flights2$I_tarde))/nrow(flights2),4)*100,"% of the complete Universe"))

[1] 9430
[1] "The 'NA's represent the: 2.8% of the complete Universe"

Option 1: Take them out of here

Option 2: Special Value

flights2_opt2; sum(is.na(flights2_opt2$I_tarde)); print(paste0("The 'NA's represent the: ", round(sum(is.na(flights2_opt2$I_tarde))/nrow(flights2_opt2),4)*100,"% of the Universe")); sum(flights2_opt2$I_tarde==-9999); print(paste0("The Special Values represent the: ", round(sum(flights2_opt2$I_tarde==-9999)/nrow(flights2_opt2),4)*100,"% of the Universe"))

[1] 0
[1] "The 'NA's represent the: 0% of the Universe"
[1] 9430
[1] "The Special Values represent the: 2.8% of the Universe"

Option N & beyond: More sophisticated stuff

print(important_mens)

[1] " We won't see any of these :D   "

Grouped operations

group_by

• Counting number of flights per day

```

References

• Go to him

• The best place to start

Cheatsheet

Two-table

Let’s go for some joins